Honestly not sure. I've never found the need to use tensorflow serving. I just simply load in the module and do predictions that way. TF serving is surely more robust for this, but I seem to get by just fine with pretty low latency without the serving. Currently enough to run through a pretty large network in about 0.003 seconds.